IE582 HW2

Çiğdem Renkli

Task 1

Part a

Apply PCA to reduce the number of dimensions to one and visualize the instances on a scatter plot. Note that the scatter plot will show the observation number versus the observed value (as we have a single feature to represent the instance).

Part b

Apply MDS to reduce the number of dimensions to one and visualize the instances on a scatter plot as in part (a). Use at least two different similarity measure.

Euclidian Distance
Manhattan Distance

Part c

On a 2D scatter plot, one can observe how two observations from two classes are different.

  1. What is your conclusion when you use PCA results in single dimension (your results from part a)?
  2. What is your conclusion when you use MDS results in single dimension (your results from part b)? Compare the results from different similarity measures used in part (b)
  3. Compare MDS results with PCA. What is your conclusion?
  1. Single dimension feature obtained from PCA seems good for classification. Using two simple linear separators (class a: feature<=1.50 and feature >= -1.10) it misclassifies only 4 observations. Accuracy for training data = 196/200

  2. Single dimension feature obtained from both MDSs seems very good for classification. For both cases using two simple linear separators (class a: feature <= 1.5 and feature >=-1), it classifies all observations correctly (training errors = 0). Both Euclidian and Manhattan distance cases looks similar but I think Manhattan distance gives slightly better results because distances between the closest points of different classes are larger.

  3. MDS result provides that all observations are classified correctly using a single dimension and two simple linear decision boundaries, while PCA misclassifies 4 of them. So MDS seems to be a better classifier for this data.

Part d

Suppose, you are not satisfied with your dimensionality reduction scheme in part (a). Add the following columns to your data, X 12 , X22 , X 1× X 2 (three columns as functions of your original variables) and apply PCA. Comment on the PCA results (i.e. what are the eigenvalues? What do they refer to?).

First apply PCA to reduce number of dimensions to 1:

Now, I do not give a spesific number of dimensions to reduce

Task 2

This is similar to Turkey map in terms of X2 axis. But it is upside down in X1 axis.

Multiplying X1 values by -1 makes plot similar to Turkey map.

Task 3

Read X

Read Y

Read Z

Part a

Calculate cumulative sums

Part b

In this setting the eigenvectors seem to imply movement along the axes